Cross-modal implicit learning of random time patterns.

Perception is sensitive to statistical regularities in the environment, including temporal characteristics of sensory inputs. Interestingly, implicit learning of temporal patterns in one modality can also improve their processing in another modality. However, it is unclear how cross-modal learning transfer affects neural responses to sensory stimuli. Here, we recorded neural activity of human volunteers using electroencephalography (EEG), while participants were exposed to brief sequences of randomly timed auditory or visual pulses. Some trials consisted of a repetition of the temporal pattern within the sequence, and subjects were tasked with detecting these trials. Unknown to the participants, some trials reappeared throughout the experiment across both modalities (Transfer) or only within a modality (Control), enabling implicit learning in one modality and its transfer. Using a novel method of analysis of single-trial EEG responses, we showed that learning temporal structures within and across modalities is reflected in neural learning curves. These putative neural correlates of learning transfer were similar both when temporal information learned in audition was transferred to visual stimuli and vice versa. The modality-specific mechanisms for learning of temporal information and general mechanisms which mediate learning transfer across modalities had distinct physiological signatures: temporal learning within modalities relied on modality-specific brain regions while learning transfer affected beta-band activity in frontal regions.

Auditory memory of complex sounds in sparsely distributed, highly correlated neurons in the auditory cortex.

Listening in complex sound environments requires rapid segregation of different sound sources e.g., speakers from each other, speakers from other sounds, or different instruments in an orchestra, and also adjust auditory processing on the prevailing sound conditions. Thus, fast encoding of inputs and identifying and adapting to reoccurring sounds are necessary for efficient and agile sound perception. This adaptation process represents an early phase of developing implicit learning of sound statistics and thus represents a form of auditory memory. The auditory cortex (ACtx) is known to play a key role in this encoding process but the underlying circuits and if hierarchical processing exists are not known. To identify ACtx regions and cells involved in this process, we simultaneously imaged population of neurons in different ACtx subfields using in vivo 2-photon imaging in awake mice. We used an experimental stimulus paradigm adapted from human studies that triggers rapid and robust implicit learning to passively present complex sounds and imaged A1 Layer 4 (L4), A1 L2/3, and A2 L2/3. In this paradigm, a frozen spectro-temporally complex 'Target' sound would be randomly re-occurring within a stream of random other complex sounds. We find distinct groups of cells that are specifically responsive to complex acoustic sequences across all subregions indicating that even the initial thalamocortical input layers (A1 L4) respond to complex sounds. Cells in all imaged regions showed decreased response amplitude for reoccurring Target sounds indicating that a memory signature is present even in the thalamocortical input layers. On the population level we find increased synchronized activity across cells to the Target sound and that this synchronized activity was more consistent across cells regardless of the duration of frozen token within Target sounds in A2, compared to A1. These findings suggest that ACtx and its input layers play a role in auditory memory for complex sounds and suggest a hierarchical structure of processes for auditory memory.

Simultaneous mnemonic and predictive representations in the auditory cortex.

Recent studies have shown that stimulus history can be decoded via the use of broadband sensory impulses to reactivate mnemonic representations.1-4. However, memories of previous stimuli can also be used to form sensory predictions about upcoming stimuli.5,6 Predictive mechanisms allow the brain to create a probable model of the outside world, which can be updated when errors are detected between the model predictions and external inputs. 7-10 Direct recordings in the auditory cortex of awake mice established neural mechanisms for how encoding mechanisms might handle working memory and predictive processes without "overwriting" recent sensory events in instances where predictive mechanisms are triggered by oddballs within a sequence.11 However, it remains unclear whether mnemonic and predictive information can be decoded from cortical activity simultaneously during passive, implicit sequence processing, even in anesthetized models. Here, we recorded neural activity elicited by repeated stimulus sequences using electrocorticography (ECoG) in the auditory cortex of anesthetized rats, where events within the sequence (referred to henceforth as "vowels," for simplicity) were occasionally replaced with a broadband noise burst or omitted entirely. We show that both stimulus history and predicted stimuli can be decoded from neural responses to broadband impulses, at overlapping latencies but based on independent and uncorrelated data features. We also demonstrate that predictive representations are dynamically updated over the course of stimulation.

Auditory memory for random time patterns in cochlear implant listeners.

Learning about new sounds is essential for cochlear-implant and normal-hearing listeners alike, with the additional challenge for implant listeners that spectral resolution is severely degraded. Here, a task measuring the rapid learning of slow or fast stochastic temporal sequences [Kang, Agus, and Pressnitzer (2017). J. Acoust. Soc. Am. 142, 2219-2232] was performed by cochlear-implant (N = 10) and normal-hearing (N = 9) listeners, using electric or acoustic pulse sequences, respectively. Rapid perceptual learning was observed for both groups, with highly similar characteristics. Moreover, for cochlear-implant listeners, an additional condition tested ultra-fast electric pulse sequences that would be impossible to represent temporally when presented acoustically. This condition also demonstrated learning. Overall, the results suggest that cochlear-implant listeners have access to the neural plasticity mechanisms needed for the rapid perceptual learning of complex temporal sequences.

Neural Correlates of Auditory Pattern Learning in the Auditory Cortex.

Learning of new auditory stimuli often requires repetitive exposure to the stimulus. Fast and implicit learning of sounds presented at random times enables efficient auditory perception. However, it is unclear how such sensory encoding is processed on a neural level. We investigated neural responses that are developed from a passive, repetitive exposure to a specific sound in the auditory cortex of anesthetized rats, using electrocorticography. We presented a series of random sequences that are generated afresh each time, except for a specific reference sequence that remains constant and re-appears at random times across trials. We compared induced activity amplitudes between reference and fresh sequences. Neural responses from both primary and non-primary auditory cortical regions showed significantly decreased induced activity amplitudes for reference sequences compared to fresh sequences, especially in the beta band. This is the first study showing that neural correlates of auditory pattern learning can be evoked even in anesthetized, passive listening animal models.

Decoding the Content of Auditory Sensory Memory Across Species.

In contrast to classical views of working memory (WM) maintenance, recent research investigating activity-silent neural states has demonstrated that persistent neural activity in sensory cortices is not necessary for active maintenance of information in WM. Previous studies in humans have measured putative memory representations indirectly, by decoding memory contents from neural activity evoked by a neutral impulse stimulus. However, it is unclear whether memory contents can also be decoded in different species and attentional conditions. Here, we employ a cross-species approach to test whether auditory memory contents can be decoded from electrophysiological signals recorded in different species. Awake human volunteers (N = 21) were exposed to auditory pure tone and noise burst stimuli during an auditory sensory memory task using electroencephalography. In a closely matching paradigm, anesthetized female rats (N = 5) were exposed to comparable stimuli while neural activity was recorded using electrocorticography from the auditory cortex. In both species, the acoustic frequency could be decoded from neural activity evoked by pure tones as well as neutral frozen noise burst stimuli. This finding demonstrates that memory contents can be decoded in different species and different states using homologous methods, suggesting that the mechanisms of sensory memory encoding are evolutionarily conserved across species.

Cortical mapping of mismatch responses to independent acoustic features.

Predictive coding is an influential theory of neural processing underlying perceptual inference. However, it is unknown to what extent prediction violations of different sensory features are mediated in different regions in auditory cortex, with different dynamics, and by different mechanisms. This study investigates the neural responses to synthesized acoustic syllables, which could be expected or unexpected, along several features. By using electrocorticography (ECoG) in rat auditory cortex (subjects: adult female Wistar rats with normal hearing), we aimed at mapping regional differences in mismatch responses to different stimulus features. Continuous streams of morphed syllables formed roving oddball sequences in which each stimulus was repeated several times (thereby forming a standard) and subsequently replaced with a deviant stimulus which differed from the standard along one of several acoustic features: duration, pitch, interaural level differences (ILD), or consonant identity. Each of these features could assume one of several different levels, and the resulting change from standard to deviant could be larger or smaller. The deviant stimuli were then repeated to form new standards. We analyzed responses to the first repetition of a new stimulus (deviant) and its last repetition in a stimulus train (standard). For the ECoG recording, we implanted urethane-anaesthetized rats with 8 × 8 surface electrode arrays covering a 3 × 3 mm cortical patch encompassing primary and higher-order auditory cortex. We identified the response topographies and latencies of population activity evoked by acoustic stimuli in the rat auditory regions, and mapped their sensitivity to expectation violations along different acoustic features. For all features, the responses to deviant stimuli increased in amplitude relative to responses to standard stimuli. Deviance magnitude did not further modulate these mismatch responses. Mismatch responses to different feature violations showed a heterogeneous distribution across cortical areas, with no evidence for systematic topographic gradients for any of the tested features. However, within rats, the spatial distribution of mismatch responses varied more between features than the spatial distribution of tone-evoked responses. This result supports the notion that prediction error signaling along different stimulus features is subserved by different cortical populations, albeit with substantial heterogeneity across individuals.

Memory for Random Time Patterns in Audition, Touch, and Vision.

Perception deals with temporal sequences of events, like series of phonemes for audition, dynamic changes in pressure for touch textures, or moving objects for vision. Memory processes are thus needed to make sense of the temporal patterning of sensory information. Recently, we have shown that auditory temporal patterns could be learned rapidly and incidentally with repeated exposure [Kang et al., 2017]. Here, we tested whether rapid incidental learning of temporal patterns was specific to audition, or if it was a more general property of sensory systems. We used a same behavioral task in three modalities: audition, touch, and vision, for stimuli having identical temporal statistics. Participants were presented with sequences of acoustic pulses for audition, motion pulses to the fingertips for touch, or light pulses for vision. Pulses were randomly and irregularly spaced, with all inter-pulse intervals in the sub-second range and all constrained to be longer than the temporal acuity in any modality. This led to pulse sequences with an average inter-pulse interval of 166 ms, a minimum inter-pulse interval of 60 ms, and a total duration of 1.2 s. Results showed that, if a random temporal pattern re-occurred at random times during an experimental block, it was rapidly learned, whatever the sensory modality. Moreover, patterns first learned in the auditory modality displayed transfer of learning to either touch or vision. This suggests that sensory systems may be exquisitely tuned to incidentally learn re-occurring temporal patterns, with possible cross-talk between the senses.

Auditory memory for random time patterns.

The acquisition of auditory memory for temporal patterns was investigated. The temporal patterns were random sequences of irregularly spaced clicks. Participants performed a task previously used to study auditory memory for noise [Agus, Thorpe, and Pressnitzer (2010). Neuron 66, 610-618]. The memory for temporal patterns displayed strong similarities with the memory for noise: temporal patterns were learnt rapidly, in an unsupervised manner, and could be distinguished from statistically matched patterns after learning. There was, however, a qualitative difference from the memory for noise. For temporal patterns, no memory transfer was observed after time reversals, showing that both the time intervals and their order were represented in memory. Remarkably, learning was observed over a broad range of time scales, which encompassed rhythm-like and buzz-like temporal patterns. Temporal patterns present specific challenges to the neural mechanisms of plasticity, because the information to be learnt is distributed over time. Nevertheless, the present data show that the acquisition of novel auditory memories can be as efficient for temporal patterns as for sounds containing additional spectral and spectro-temporal cues, such as noise. This suggests that the rapid formation of memory traces may be a general by-product of repeated auditory exposure.

Acquisition versus consolidation of auditory perceptual learning using mixed-training regimens.

Learning is considered to consist of two distinct phases-acquisition and consolidation. Acquisition can be disrupted when short periods of training on more than one task are interleaved, whereas consolidation can be disrupted when a second task is trained after the first has been initiated. Here we investigated the conditions governing the disruption to acquisition and consolidation during mixed-training regimens in which primary and secondary amplitude modulation tasks were either interleaved or presented consecutively. The secondary task differed from the primary task in either task-irrelevant (carrier frequency) or task-relevant (modulation rate) stimulus features while requiring the same perceptual judgment (amplitude modulation depth discrimination), or shared both irrelevant and relevant features but required a different judgment (amplitude modulation rate discrimination). Based on previous literature we predicted that acquisition would be disrupted by varying the task-relevant stimulus feature during training (stimulus interference), and that consolidation would be disrupted by varying the perceptual judgment required (task interference). We found that varying the task-relevant or -irrelevant stimulus features failed to disrupt acquisition but did disrupt consolidation, whereas mixing two tasks requiring a different perceptual judgment but sharing the same stimulus features disrupted both acquisition and consolidation. Thus, a distinction between acquisition and consolidation phases of perceptual learning cannot simply be attributed to (task-relevant) stimulus versus task interference. We propose instead that disruption occurs during acquisition when mixing two tasks requiring a perceptual judgment based on different cues, whereas consolidation is always disrupted regardless of whether different stimulus features or tasks are mixed. The current study not only provides a novel insight into the underlying mechanisms of perceptual learning, but also has practical implications for the optimal design and delivery of training programs that aim to remediate perceptual difficulties.